Method and apparatus for generating a stereographic image

ABSTRACT

In a 3D image generating apparatus  1  depth values (Z-value) of pixels are calculated on the basis of grayscale values of 2D image data. Next, a reverse determination, smoothing, distribution adjustment, local adjustment, and the like are performed on the Z-values. Next, parallaxes of pixels are determined on the basis of the Z-values of the pixels finally obtained, and the pixels are displaced acceding to the parallaxes, so as to generate data of a stereographic image.

TECHNICAL FIELD

The present invention relates to a method and apparatus for generatingthree-dimensional image data by means of two-dimensional image data.

BACKGROUND ART

Humans are able to visually perceive three-dimensional objects inthree-dimensional space. This ability is known as stereopsis. Stereopsisoccurs when a human brain recognizes an object in three-dimensions onthe basis of two images, received by the right and the left eyes,respectively. In other words, it is necessary for each of the right andthe left eyes to receive an image for stereopsis to occur.

A technique called stereography has been developed, in which imageprocessing is performed on a two-dimensional image so that a viewer canperceive the processed two-dimensional planar image as athree-dimensional image. Stereography involves an image processingtechnique of revising image data of a single two-dimensional planarimage to generate image data for each of the right and left eyes.Specifically, the image data for each of the right and left eyes isgenerated by calculating a binocular parallax with regard to an object(subject) depicted in an image and displacing, according to theparallax, positions of pixels (dots) of the image in right and leftdirections, respectively.

For example, the parallax is determined by a value representative of thedegree of a farness (or nearness) of an object (hereinafter referred toas a Z-value), which is calculated on the basis of at least one ofsaturation, brightness, and hue of the object. For example, Japanesepatent JP2002-123842A discloses a method for determining a Z-value onthe basis of saturation and brightness of the original 2D image.Specifically, the method includes: (a) a step of determining Z-valuesfor each pixel on the basis of saturation of the image data; (b) a stepof determining a contour line(s) of the object(s) included in the image;(c) a step of calculating an average of the Z-values of all pixelswithin the object, and (d) a step of determining that the average is aZ-value of the object. A parallax with respect to the object isdetermined on the basis of the Z-value of the object.

In addition, to improve perception of the generated 3D image, saturationof pixels near the object is changed so as to “blur” an edge of theobject. It is to be noted that such a blurring process depends on anempirical rule, that is, the fact the farther away an object is locatedin an image, the fainter its image is (namely, the lower its saturationis), and the nearer the object is located, the sharper its image is(namely, the higher its saturation is). This is because edges of distantobjects tend to be fine due to dust in the air and the like, blurringthe object. Nearby objects increase in number and in saturation, andedges of the objects are thicker and thus look sharper.

However, there are images that do not follow the above rule, such as animage in which a bright sky is depicted at a far distance and adark-colored forest at a near distance. In such a case, the rule onwhich a Z-value is calculated does not represent a true depth ofobjects. As a result, it is not possible to reproduce a correct parallaxwith the generated stereographic image, and a viewer is thus given anunnatural impression.

Further, in a case where a shadow or a highlight hangs over a certainportion of an object, in other words in a case where a variation ofsaturation within the object is considerable, it is not possible tocalculate a true Z-value of the object. As a result, a generatedstereographic image gives a viewer an impression that only theoverhanging portion is projected (or recessed) locally. In this case,since it is not possible to determine an edge of objects precisely, theblurring process cannot contribute to improvement of plasticity of the3D image.

Still further, in a highlighted portion of an object in which a Z-valuebecomes larger, a parallax of that portion is overestimated accordingly.As a result, a phenomenon will occur in which adjacent pixels in anoriginal 2D image are displaced in opposite horizontal directions whengenerating a stereographic image. This phenomenon is called a pixelcrossing, generating a distorted portion in the generated stereographicimage. As a result, a quality of a generated stereographic image isreduced to such an extent that it cannot be appreciated by a viewer.Occurrence of this phenomenon is especially serious at a position wherea number of objects overlap. At such a position, distortion of the imagecan occur easily; thus it becomes significantly difficult to achievenatural perception of a stereographic image for a viewer.

As is described above, since it is not possible in the prior art tocalculate a correct Z-value from an original image data to generate astereographic image, generation of a realistic stereographic image isdifficult.

SUMMARY OF THE INVENTION

The present invention has been made, taking into account the abovebackground art, to provide a method for generating stereographic imagedata that gives a viewer an impression of natural plasticity, and anapparatus for generating stereographic image data adapted to the method.The present invention is a result of the applicant's continuing studieson necessary conditions for giving a human a strong impression ofplasticity.

A method for generating a stereographic image according to the presentinvention is characterized by comprising: a calculating step ofcalculating Z-values of pixels on the basis of image data of pixels, thepixels forming an image, each Z-value being assigned to a pixel, andeach Z-value representing a depth of an object corresponding to thepixel; an adjusting step of adjusting a Z-value of a target pixelobtained in the calculating step using a Z-value of a pixel other thanthe target pixel; and a generating step of determining an amount ofdisplacement of a target pixel on the basis of an adjusted Z-value ofthe target pixel, and displacing the target pixel horizontally by thedetermined amount, to generate images for the right and the left eyes.By adjusting Z-values, a stereographic image which provides a viewerwith natural plasticity is generated.

In a preferred embodiment, in the calculating step the weights aredetermined based on the ratio of cone cells sensitive of R, G, and B,respectively, which cones exist in a retina of a human eye.

It is possible that in the adjustment step Z-values of pixels areadjusted so that a single step available for a Z-value of a pixelcorresponding to an object located backward in an original image expressdeeper depth than a single step available for a Z-value of a pixelcorresponding to an object located forward in the original image.

It is possible that in the adjusting step an average of Z-values ofpixels within an area which includes a target pixel is obtained and aZ-value of the target pixel is replaced by the obtained average.

Further, it is possible that in the adjusting step a distribution of theZ-values of all pixels in the image and an average of all pixels in theimage are obtained and deviation of the distribution is corrected usingthe obtained average.

Still further, it is possible that in the adjusting step at least oneobject in the image represented by the image data is identifiedreferring to Z-values of pixels calculated in the calculating step and aZ-value of the target pixel is adjusted on the basis of a Z-value of apixel located within an area corresponding to the identified object.

Still further, it is possible that in the adjusting step any one of astep size of quantization of the Z-value, and an upper limit and a lowerlimit of the calculated Z-value is determined based on a value of aparameter specified by a user.

Still further, it is possible that in said adjustment step tendency ofZ-values of pixels in the image is analyzed by comparing a Z-value of apixel within an area with a Z-value of a pixel within another area, andwhen a result of the analysis agrees with a predetermined condition, aquantitative relation between the amount of displacement of the targetpixel and the Z-value of the target pixel is reversed in the generatingstep.

In a preferred embodiment, it is possible that the stereographic imagegenerating apparatus further comprises a step of obtaining moving imagescomprised of a plurality of images, and wherein a stereographic image isgenerated from each image, to generate stereographic imagescorresponding to the moving images in real time.

A stereographic image generating apparatus for generating astereographic image according to the present invention is characterizedin comprising: a calculating means for calculating Z-values of pixels onthe basis of image data of pixels, the pixels forming an image, eachZ-value being assigned to a pixel, and each Z-value representing a depthof an object corresponding to the pixel; an adjusting means foradjusting a Z-value of a target pixel obtained in the calculating meansusing a Z-value of a pixel other than the target pixel; and a generatingmeans for determining an amount of displacement of a target pixel on thebasis of an adjusted Z-value of the target pixel, and displacing thetarget pixel horizontally by the determined amount, to generate imagesfor the right and the left eyes. It is possible that the apparatusfurther comprises an obtaining means for obtaining from a user aparameter used in said adjusting means.

The present invention also provides a computer program for functioning acomputer as the stereographic image generating apparatus. It is possiblethat the program is stored in a various types of storage media and isdownload via a network, so as to be installed in the computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a 3D imagegeneration apparatus according to the first embodiment of the presentinvention.

FIG. 2 is a schematic diagram illustrating an example of a screendisplayed by a 3D image drawing application.

FIG. 3 is a schematic diagram illustrating an example of a configurationof a setting dialogue panel.

FIG. 4 is a schematic diagram illustrating a mechanism of displaying a3D image according to the first embodiment of the present invention

FIG. 5 is also a schematic diagram illustrating a mechanism ofdisplaying a 3D image according to the first embodiment of the presentinvention.

FIG. 6 illustrates anaglyph glasses.

FIG. 7 is a flowchart showing image processing performed when a 3D modeis selected.

FIG. 8 is a schematic diagram illustrating a format of data of an image.

FIG. 9 is a flowchart showing a grayscale value adjustment.

FIG. 10 is a flowchart showing the grayscale value adjustment (cont'd).

FIG. 11 is a flowchart showing the grayscale adjustment (cont'd).

FIG. 12 is a diagram illustrating sampling areas and correspondingweight coefficients used in a reverse determination.

FIG. 13 is a flowchart showing the reverse determination.

FIG. 14 is a flowchart showing the reverse determination (cont'd).

FIG. 15 is a diagram illustrating matrix areas MMs used in smoothing.

FIG. 16 is a flowchart showing the smoothing.

FIG. 17 is a conceptual diagram illustrating a distribution adjustment.

FIG. 18 is a flowchart showing the distribution adjustment.

FIG. 19 is also a flowchart showing the distribution adjustment(cont'd).

FIG. 20 illustrates a distribution of Z-values with regard to an objectadapted to a local adjustment.

FIG. 21 is a flowchart showing the local adjustment.

FIG. 22 is a flowchart showing a contour determination included in thelocal adjustment.

FIG. 23 is a flowchart showing the contour determination (cont'd).

FIG. 24 is a flowchart showing the contour determination (cont'd).

FIG. 25 is a flowchart showing an object center determination includedin the local adjustment.

FIG. 26 is a flowchart showing the object center determination (cont'd).

FIG. 27 is a flowchart showing the object center determination (cont'd).

FIG. 28 is a flowchart showing an intersection determination included inthe local adjustment.

FIG. 29 is a flowchart showing an inplane Z-value calculation includedin the local adjustment.

FIG. 30 is a flowchart showing an inplane Z-value correction.

FIG. 31 is a flowchart showing a Z-value adjustment.

FIG. 32 is a flowchart showing a pixel displacement.

FIG. 33 is a flowchart showing the pixel displacement (cont'd).

FIG. 34 is a flowchart showing the pixel displacement (cont'd).

FIG. 35 illustrates an enhancement according to a modified embodiment ofthe present invention.

FIG. 36 is a flowchart showing an overview of image processing accordingto the modified embodiment.

FIG. 37 is a flowchart showing an enhancement according to the modifiedembodiment.

FIG. 38 is a conceptual diagram illustrating a scheme of displaying astereographic image according to the modified embodiment.

FIG. 39 is also a conceptual diagram illustrating a scheme of displayinga stereographic image according to the modified embodiment.

FIG. 40 is also a conceptual diagram illustrating a scheme of displayinga stereographic image according to the modified embodiment.

FIG. 41 is also a conceptual diagram illustrating a scheme of displayinga stereographic image according to the modified embodiment.

FIG. 42 is a flowchart showing a pixel displacement according to themodified embodiment.

FIG. 43 is a block diagram illustrating a functional configuration of aMobile terminal 1A of the second embodiment of the present invention.

FIG. 44 is an external view of Mobile terminal 1A.

FIG. 45 is a schematic diagram illustrating a configuration of a display21 b.

FIG. 46 illustrates a principle of exciting stereopsis when an image isviewed by the naked eye.

FIG. 47 is a flowchart showing a process for displaying a static imagepreformed by Mobile terminal 1A.

FIG. 48 is a flowchart showing a process for displaying moving imagespreformed by Mobile terminal 1A.

FIG. 49 is a block diagram illustrating an example of a functionalconfiguration of a 3D image generating apparatus 1B according to thethird embodiment of the present invention.

FIG. 50 is a flowchart showing a process for displaying moving imagespreformed by 3D image generating apparatus 1B.

FIG. 51 is a block diagram illustrating a functional configuration of atrain simulator 40 according to an application of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Detailed description of the preferred embodiments of the presentinvention will now be described with reference to the accompanyingdrawings.

FIRST EMBODIMENT

<A. Configuration of the Apparatus>

FIG. 1 shows a 3D image generating apparatus 1 of the first embodimentof the present invention. In FIG. 1, a CPU (central processing unit) 11controls all units of 3D image generation apparatus 1 to performprocessing including generation of stereographic image data. A ROM (readonly memory) 12 is employed to store programs, one of which is executedwhen the apparatus is turned on, and related data. A RAM (random accessmemory) 13 is employed as a work area of CPU 11.

A Character generator 14 generates font data to be displayed on adisplay. A Time circuit 15 outputs a current time. An Interface 16connects 3D image generation apparatus 1 to the Internet (not shown). ATransmission control unit 17 transmits and receives data to and fromterminals connected to the Internet.

A Magnetic disk drive 18 stores programs for performing image processingexecuted by 3D image generation apparatus 1, including an OS program anddata generated in the image processing steps. A CD-ROM drive 19 isemployed to read out image data stored in a CD-ROM 20. A Display 21includes a CRT, LCD or the like, which displays an image under controlof a Display control unit 22.

A Keyboard 23 is a standard full-sized keyboard which enables a user toinput instructions. A Pointing device 24 is an input device such as atouch panel for a user to input instructions. An Input control unit 25obtains instructions input via Keyboard 23 and Pointing device 24 to beoutput to CPU 11.

CPU 11, ROM 12, RAM 13, Character generator 14, Time circuit 15,Transmission control unit 17, Magnetic disk drive 18, CD-ROM drive 19,Display control unit 22, and Input control unit 25 are connected to anInternal bus 26. Thus, data is transferred between those units viaInternal bus 26.

In generating 3D image data CPU 11 executes an application fordisplaying a 3D image, the application being stored in ROM 12. Anexample of a screen displayed in Display 21 when the application isrunning is depicted in FIG. 2.

As shown in FIG. 2, the screen displayed when the application is runningis comprised of a main screen MD and a menu bar MB which includes menuitems. Moving images selected by a user are displayed in Main screen MD.

Items MI through MI10 are displayed in Menu bar MB. Item MI1 is selectedfor identifying an image on which a user wishes to perform imageprocessing. Item MI2 is selected for setting display preferences of astereographic image. Item MI3 is selected for storing data of astereographic image. Item MI4 is selected for terminating theapplication. Item MI5 is selected for starting to display moving imageson Main screen MD. Item MI6 is selected for terminating displayingmoving images on Min screen MD. Item MI7 is selected for pausing displayof moving images in Min screen MD.

Item MI8 shows a mode (either normal (2D) mode or stereogram (3D) modeof moving images displayed on Main screen MD), and is selected forswitching these modes. Specifically, MI8 is changed between normal (2D)and stereogram (3D) modes when selected by a user. The upper and thelower figures of FIG. 2 represent a screen when 2D mode and 3D mode arein effect, respectively.

Item MI9 is selected for specifying a mode of effecting a combiningscheme (either odd/even mode or anaglyph mode). It is assumed in thisembodiment that only these two schemes are available. However, it ispossible to add other modes corresponding to other combining schemes foreffecting stereopsis. Specifically, in combining images to obtain asingle stereographic image, it is possible to employ a combining schemein which four images corresponding to four viewpoints are used. Further,it is possible to employ a stereopsis scheme in which eight or nineimages corresponding respectively to eight or nine viewpoints which areselected by a user are used.

Item MI10 indicates a mode of effecting a projection direction, one ofwhich is an F1 mode that gives a viewer of the stereographic image avisual impression that a three-dimensional image is recessed relative tothe surface of the screen, and the other of which is an F2 mode thatgives a viewer of the stereographic image a visual impression that athree-dimensional image is projected frontward relative to the surface,and is used for switching these mode. In both screens depicted in FIG. 2F1 mode is selected; thus, a viewer will perceive that athree-dimensional image is recessed relative to the surface.

FIG. 3 shows an example of a dialogue panel displayed in display 21 whena user clicks Item MI2. As shown in FIG. 3, a Title bar BR is located inthe upper side of the screen, in which a Close box BC for causing thisdialogue panel to close is located.

Sliders IS1 and IS2, Input boxes IB1, IB2, IB3, and IB4, Check boxes IB5and IB6, and buttons BM2 and BM3 are arranged in the setting dialoguepanel. A user can operate slider IB1 to specify a value of an interfacedepth. The value of “an interface depth” specified by the user isdisplayed in Input box IB1. It is possible for the user to input anumber directly in Input box IB1.

It is to be noted that interface depth represents a perceived degree ofbeing recessed (or projected frontward) of an image relative to thesurface of the screen. Thus, the larger the interface depth is, the morethe three-dimensional world is perceived to be projected (or recessed).For example, a stereographic image which is generated by combining animage for the left eye of a user in which pixels are displaced leftwardon the basis of a parallax, and an image for the right eye of the userin which pixels are displaced rightward is perceived by the user as ifthe world is recessed relative to the surface; if the pixels of bothimages are moved in the opposite directions, a user will perceive thatthe stereographic image is projected frontward.

A user can move Slider IS2 to specify a value of “undulation”. The valuespecified by the user is displayed in Input box IB2. It is possible fora user to directly enter a number in Input box IB2. It is to be notedthat the undulation indicates a difference of altitudes between thefarthest (deepest) point and the nearest (shallowest) point in astereographic image. Thus, the bigger the undulation is, the more theundulation is emphasized. On the contrary, a user perceives that theworld is flatter with a smaller undulation.

A user can enter the maximum and the minimum values of “an object depth”in Input boxes IB3 and IB4, respectively. An object depth is assignedfor a single object included in an image, which is a parameterrepresentative of a position of the object in perceived depth (thedegree of being projected frontward or backward relative to the surfaceof the screen). Button BM1 is used for setting the maximum and theminimum values of an object depth to a default value.

A user can specify a first line to be depicted (either an odd-numberedline or an even-numbered line) by checking either a Check box IB5 orIB6. When the button BM2 is pressed, the current settings are set. Whenthe button BM3 is pressed, the current settings are cancelled and theprevious setting is set. Parameters set via this dialogue panel are usedin image processing which will be described later.

There are various schemes using a pair of images for effectingstereopsis for a viewer. In this embodiment, a so-called odd-even schemeshould be adopted. In this case, with a polarizing stripe filter placedon a screen of display 21, a user wearing polarized glasses is able toperceive an image on the screen three-dimensionally.

More specifically, as shown in FIG. 4, Polarized stripe filters for theleft eye FL0, FL2, . . . , having a predetermined width and length areplaced, at regular intervals, over even-numbered lines of the horizontalscanning lines of the screen SL0, SL2, . . . , respectively. Over theodd-numbered lines of the screen SL1, SL3, . . . , Polarizing filtersFL1, FL3 having a predetermined width are placed at regular intervals,respectively. As shown in FIG. 5, image data for the left eye isdisplayed in the even-numbered lines of Display 21 SL0, SL2, . . . , andimage data for the right eye is displayed in the odd-numbered lines ofDisplay 21 SL1, SL3, . . . . A polarization direction of Polarizationstripe filters for the left eye FL0, FL2, . . . are orthogonal to adirection of polarization stripe filters for the right eye FL1, FL3, . .. .

As shown in FIG. 6, a filter for the left eye Fleft and a filter for theright eye Fright are placed in Polarized glasses GS. Polarizationdirections of Fleft and Fright are the same as the directions of thepolarization filters for the left and the right eye, respectively.

When a user watches the screen of Display 21 wearing Polarized glassesGS, light originated at the even-numbered lines of Display 21 SL0, SL2,. . . are transmitted through Polarizing filters FL0, FL2, . . . ; thus,the light reaches the left eye of the viewer through the filter Fleft,but does not reach the right eye of the viewer since the filter Frightcuts the light. On the contrary, light originated at the odd-numberedlines of Display 21 SL1, SL3, . . . are transmitted through Polarizingfilters FL1, FL3, . . . , and then reach the right eye of the viewer,but the light does not reach the left eye of the viewer with the filterFleft. Thus, images intended for the left and the right eyes aresubjected to the left and right eye, respectively, thereby enabling theviewer to perceive a displayed image three-dimensionally.

<B. Image Processing>

FIG. 7 shows a flow of an image processing performed by 3D imagegenerating apparatus 1 in a case where a user has performed apredetermined operation to select 3D mode when the applicationdisplaying a 3D image is running.

At first, CPU 11 reads data of a single frame of color moving images(step S101), and checks whether the data is in RGB format (step S102).In a case where the data is not in RGB format, CPU 11 converts the datato RGB format (step S103). The data of color moving images can beformatted in any type of formats. The data may be compressed. In a casewhere the data is compressed, decompression is employed before stepS103. Color depth (bit depth) for a pixel of color moving image data maybe 16, 24, or 32 bits.

Further, the color moving image data may be stored on Magnetic disk 18or CD-ROM 20. Still further, the data may be input via a cable from animage reproducing device such as a DVD player. Still further, the datamay be image data of a TV program broadcast via a satellite or cable, tobe supplied from a TV set connected to 3D image generating apparatus 1.Still further, the data may be downloaded from a server on the Internet.

Next, CPU 11 calculates grayscale values of pixels on the basis of imagedata value (R value, G value, B value), and assigns the grayscale valuesto Z-values of the pixels (step S104, hereinafter referred to as agrayscale value adjustment).

After Z-values are determined for all the pixels, Z-values are adjustedthrough reverse determination (step S105), smoothing (step S106),distribution adjustment (step S107), and local adjustment (steps S108and S109). Finally, the degree of displacement of a pixel is calculatedon the basis of Z-values using the above adjustments. A singlestereographic image is generated by displacing pixels (step S110). Inthe following, the adjustment processes are described in detail.

In the following description, a method for describing syntaxes anddeclaration of data types is assumed to be in conformity with C Languageor an extension thereof (C++ etc.). In addition, in this embodiment,color image data is formatted by a color depth of 16, 24, or 32 bits, asdescribed above. As shown in FIG. 8, in a case of 16 bits color imagedata, each pixel has data of 5 bits, 6 bits, and 5 bits corresponding toR, G, and B, respectively. In a case of 24 bits data, each pixel hasdata of 8 bits, 8 bits, and 8 bits corresponding to R, G, and B,respectively. In a case of 32 bits data, each pixel has data in whichdummy data is added ahead of the 24 bits color image data.

<B-1. Grayscale Value Adjustment>

Grayscale value adjustment shown in Step S104 of FIG. 7 will now bedescribed with reference to FIGS. 9, 10 and 11.

In the grayscale value adjustment, grayscale components of R, G, and Bare modified using certain weights. Specifically, according to Equation(1) predetermined weights are respectively assigned to R, G, and B colorcomponents of each pixel, to revise a grayscale value.Z=0.3*R+0.59*G+0.11*B  (1)

In a case of 16 bits data, available values for Z-value of a pixel rangefrom 0 to 31. In a case of 24 or 32 bits data, the values range from 0to 255. It is to be noted that the above weights 30%, 59%, and 11% aredetermined based on the ratio of cone cells sensitive to R, G, and B,respectively, which exist in the retina of a human. Although otherweights may be employed, a stereographic image generated on the basis ofa grayscale obtained by the above weights can give a viewer the mostnatural plasticity.

Next, a weight-adjusted grayscale value is further adjusted. If thegrayscale value per se is employed to express a depth, in which a singlestep gradation corresponds to a unit of depth, both a gradation stepwidth for a pixel corresponding to an object located in the frontwardarea and a gradation step width for a pixel corresponding to an objectlocated in the backward area in an original 2D image represent the depthdifference. In other words, such a grayscale value cannot correctlyexpress a depth of an object in an original image. It is thereforerequired to revise a grayscale step so that a single step of a grayscalefor a pixel corresponding to an object located in the backward area canexpress farther depth than a single step for a grayscale of a pixelcorresponding to an object located in the frontward area. Specifically,a series a_(n) (n=1, 2, 3 . . . ) is employed to represent grayscales ofpixels corresponding to different depths. Z-values of pixels aremodified according to an arithmetic series according to Equation (2) ora geometric series according to Elution (3), to assign a bigger depthunit to a pixel corresponding to an object located in the backward areathan to a pixel corresponding to an object located in the frontwardarea.a _(n) =a+(n−1)*d  (2)a _(n) =a*r ^(n−1)  (3)where n is a positive integer, a is a first term, and the commondifference d and the common ratio r, which represents correctionfactors, can be determined based on characteristics of the original 2Dimage.

In the following description, *src is a pointer representative of anaddress in which original image data src is stored, WD represents thenumber of pixels horizontally lined in the original 2D image, HTrepresents the number of pixels vertically lined in the original image,and *zinfo is a pointer representing an address to store a calculatedZ-value. For example, if a data size of the original 2D image is 1024dots by 768 dots, WD is 1024 and HT is 768.

First, as shown in FIG. 9, CPU 11 determines whether image data to beprocessed has a color depth of 16 bits (Step S201). When the color depthis 16 bits, a variable declaration unsigned short *srcp is carried out(Step S202), coercion of data of a target pixel is performed to obtaindata in a form unsigned short (Step S203), and an address of the datasrc is assigned to the variable srcp (step S204).

Next, CPU 11 resets a value of a counter CC to 0 (Step S205). In StepS206 CPU 11 determines whether the value of the counter CC is smallerthan WD*TH (Step S206). If the condition of Step S206 is not satisfied,CPU 11 determines that processing with respect to a an entire screen ofpixels is completed, to end the grayscale value adjustment. If thecondition of Step S206 is satisfied, grayscale value adjustment isperformed in the following Steps S207 through S210. Specifically, inStep S207, with respect to R value, CPU 11 conducts an AND operationbetween 16 bits data stored in an area specified by the pointer *srcpand a hexadecimal F800 or 1111100000000000 in binary, performs anarithmetic shift right by five bits, and then multiplies by 30. Next CPU11 performs a coercion with respect to the value calculated above so asto obtain an updated R value in int type (Step S207). Thus far, a valueof the original R value multiplied by 30 is assigned to the variable R.In Step S208, with respect to G value, CPU 11 conducts an AND operationbetween 16 bits data stored in an area specified by the pointer *srcpand a hexadecimal 07E0, performs an arithmetic shift right by 6 bits,and then multiplies by 59. Next CPU 11 performs a coercion with respectto the value calculated above, so as to obtain an updated G value in inttype (Step S208). As a result, a value of the original G valuemultiplied by 59 is assigned to the variable G.

In Step S209, with respect to the B value, CPU 11 conducts an ANDoperation between 16 bits data stored in an area directed by the pointer*srcp and a hexadecimal 001F, performs an arithmetic shift right by 5bits, and then multiplies by 11. Next CPU performs a coercion withrespect to the value calculated above, so as to obtain an updated Gvalue in int type (Step S209). As a result, a value of the original Gvalue multiplied by 11 is assigned to the variable B. Finally, CPU 11assigns the sum of variables R, G, and B divided by 100 to an addressidentified by a pointer *zinfo (Step S210).

Thus far, processing with respect to a single pixel is finished, thevariables srcp, zinfo, and a current value of the counter CC areincremented by 1 (Step S211), and the processing goes back to Step S206to perform the above processing on another pixel. This processing issimilarly performed on all the pixels within a screen.

In a case where the condition of Step S201 is not satisfied, CPU 11further determines whether the image data is 24 bits formatted (StepS215 of FIG. 10). Next, as in Steps S205 and S206 shown in FIG. 9, thecounter is reset (Step S216) and the determination of Step S217 iscarried out.

Next, R, G, and B values are adjusted in a manner similar to a case of16 bits data. Specifically, instead of Steps S207 through 209 shown inFIG. 9, Steps S218 through S220 shown in FIG. 10 are carried out.Specifically, as for the variable B, 24 bits data stored in an addressspecified by a pointer *src is multiplied by 11 to assign to thevariable B (Step S218). Next, 24 bits data stored in an addressspecified by a pointer *(src+1) is multiplied by 59 to assign to thevariable G (Step S219). Next, 24 bits data stored in an addressspecified by a pointer *(src+2) is multiplied by 30 to assign to thevariable R (Step S220). The subsequent steps, including Steps 221 and222, are similar to a case of 16 bits data.

If the condition of Step S215 is not satisfied, CPU 11 determines that acolor depth of the data is 32 bits data. In this case, a variabledeclaration unsigned long *srcp is performed (Step S225 of FIG. 11), anda type of data of a target pixel is converted to unsigned long (StepS226), and an address of data src is assigned to the variable srcp (StepS227). Next, similarly to a case of 16 bits data , the counter is reset(Step S228) and the terminate check is performed in Step S229.

In Steps S230 through S232, R, G, and B values are adjusted.Specifically, an AND operation between 32 bits data stored in an addressspecified by a pointer *srcp and a hexadecimal 00FF0000 is conducted,and an arithmetic shift right by 16 bits is carried out and thenmultiplied by 30. Data type of the value is converted to int type, andfinally the value is assigned to the variable R (Step S230). As aresult, a value of the original 8 bits R value multiplied by 30 isassigned to the variable R.

Next, CPU 11 conducts an AND operation between 32 bits data and ahexadecimal 0000FF00, performs an arithmetic shift right by 8 bits, andthen multiplies it by 59, so as to assign original G value multiplied by59 to the variable G. As a result, G value multiplied by 11 is assignedto the variable B. Next, CPU 11 conducts an AND operation between 32bits data stored in an area specified by the pointer *srcp and ahexadecimal 000000FF, and multiplies it by 11. This data is converted toint type data to be assigned to the variable B (Step S232). As a result,B value multiplied by 11 is assigned to the variable B. Subsequent stepsare similar to a case of 16 and 24 bits data.

<B-2. Reverse Determination>

Referring to FIGS. 12 through 14, the reverse determination of Step S105of FIG. 7 will now be described. In the reverse determination, a Z-valuecalculated in Step S104 is adjusted so as to trigger natural plasticity.

As described above, it is enough, in principle, for generating astereographic image only to determine the amount of displacement of apixel on the basis of a Z-value of the pixel. The amount of displacementis, for example, determined in direct proportional to a Z-valuecalculated in Step S104. Accordingly, a viewer of a stereographic imageperceives that an object having a larger Z-value (“brighter” object) islocated in the near side and an object having a smaller Z-value(“dimmer” object) is located in the far side. Thus, when a dimmer objectis actually disposed in a far side and a brighter object is actuallydisposed in a near side in an original 2D image, a 3D image generatedfrom that 2D image will successfully trigger natural and realisticplasticity.

However, there is a case where an original 2D image which includes anobject having a large Z-value should be disposed in the far side and anobject having a small Z-value should be disposed in the near side; forexample, when a dim object is disposed in a bright background (e.g. bluesky). In this case, if the amount of displacement is simply determinedin direct proportion to a Z-value of the pixel, a generated 3D imagewill trigger unnatural plasticity since the positional relation ofobjects between far and near is reversed.

In light of the above, in the reverse determination, tendency ofZ-values in an original 2D image is analyzed, and whether to reverse aqualitative relation between the amount of displacement and a Z-value isdetermined on the basis of the deviation. Specifically, as shown in FIG.12, a screen displaying data of an original 2D image is divided into theupper left, upper center, upper right, middle left, middle center,middle right, lower left, lower center, and lower right areas SS1 thoughSS9 (hereinafter referred to as “sampling areas”), each of which is 8pixels in width and 8 pixels in length. It is possible to modify thesize and the number of the sampling areas.

Each sampling area has a weight. Specifically, 3, 4, 3, 2, 5, 2, 1, 2,and 1, are assigned to areas SS1, SS2, SS3, SS4, SS5, SS6, SS7, SS8, andSS9, respectively. Although it is possible to employ other weights, theweights are determined on the basis of a general tendency relating to anobject in an image. “General tendency” is an empirical rule that theupper portion of an image is more likely to include a background (e.g. ablue sky) than the lower portion of the image, and that, in a case wherean image is a picture including, for example, a man or a flower, anobject is usually disposed in the center of the image. According to thegeneral tendency, it is preferable to assign the largest weight to acentral area and larger weights to the upper areas than to the lowerareas.

Referring to FIGS. 13 and 14, a flow of the reverse determination willbe described in detail. At first CPU 11 calculate an average of Z-valuesof all the pixels of data of an image (Step S301). Next, CPU 11 resetsvalues of variables S1 and S2, and a counter CT to 0 (Step S302). Thevariables S1 and S2 are parameters that represent weights of areashaving weights larger than and smaller than an average weight of all thepixels in the screen, respectively. A value of the counter CT representsthe number of sampling areas for which the reverse determinations havealready been performed.

Next, it is checked whether a current value of the counter CT is smallerthan 9 (Step S303), to determine whether the reverse determination hasbeen performed on all of the sampling areas SS. If the condition of StepS303 is satisfied, whether an average of Z-values of pixels within atarget sampling area is larger than the average value MZ is checked(Step S304).

If the condition of Step S304 is satisfied, a weight of the CTthsampling area is added to the variable S1 (Step S305). If the conditionof Step S304 is not satisfied, the weight of the CTth sampling area isadded to the variable S2 (Step S306). Next, a value of the counter CT isincremented by 1 (Step S307), and the process goes back to Step S303 toperform the processing on another sampling area.

When the reverse determination is finished for all the 9 sampling areas(Step S303, NO), the processing goes to Step S308 of FIG. 14, in whichit is checked whether the variable S1 is larger than the variable S2. Ina case where S1>S2 (Step S308, YES), a qualitative relation between theamount of displacement and the Z-value is reversed (Step S309). In acase where the condition of Step S308 is not satisfied (Step S308, NO),it is checked whether S1 is smaller than S2 (Step S310). In a case whereS1<S2 (Step S310, YES), the qualitative relation is not reversed (StepS311). In a case where S1>S2 and S1<S2, namely, S1=S2 (Step S310, NO),it is checked whether an average of the Z-values of the sampling areaSS5 is larger than the average MZ (Step S312). If the condition of StepS312 is satisfied, the relation is not reversed (Step S313); if thecondition is not satisfied, the relation is reversed (Step S314).

It is possible to employ another check as described below. Specifically,a sampling area(s) in which an average Z-value of pixels within the areais larger than an average Z-value of all pixels included in the image isspecified, and the sum of the weights of the specified areas isobtained. If the sum is larger than a predetermined threshold, therelation between the amount of displacement and the Z-values isreversed, since the original 2D image has a tendency in which an objectdisposed in the near side is dimmer and an object disposed in the farside is brighter. If the sum is lower than the predetermined threshold,the relation is maintained.

Next, CPU 11 checks whether the item MI1 of the application fordisplaying a 3D image is set to F2 mode (frontward projection view)(Step S315). When F2 mode is effected, the relation which has or has notbeen reversed in Steps S309, S311, S313, and S314 is reversed or furtherreversed (Step S316). This is because when F2 mode is effective, it isrequired to reverse the relation between depths and Z-values. At thisstage, the reverse determination is completed. The final determinationas to whether to effect reverse of the relation, which is obtainedthrough the reverse determinations, is reflected in the subsequentprocessing in Step S110 of FIG. 7 which will be described later.

<B-3. Smoothing>

Subsequent to the reverse determination, CPU 11 performs processing forsmoothing deviations of Z-values calculated in Step S104 of FIG. 7 (StepS106). The reason for performing this process will now be described.

As described above, a stereographic image is generated by shiftingpixels horizontally by an amount corresponding to their parallaxesdetermined by their Z-values. Accordingly, a difference between aparallax of a pixel and that of the neighboring pixels will be large inan area which includes pixels having a considerable difference inZ-values. In that area, ‘pixel crossing phenomenon’ will arise in anarea where the neighboring pixels are shifted in opposite directions.This phenomenon tends to appear especially at an edge of a highlightedobject. When the phenomenon appears, blur of the edge, halation,interference fringe or the like will appear in a generated stereographicimage. The smoothing is performed to adjust a Z-value so as to preventthose abnormalities.

Specifically, as shown in FIG. 15, CPU 11 calculates for a pixel Px anaverage of Z-values included in an original 2D image, and an average ofZ-values of the pixels in a predetermined region MM (hereinafterreferred to as a matrix), the region including the pixel Px. CPU 11 thenupdates the calculated average for a Z-value of the pixel Px.Accordingly, the Z-value of the pixel Px is assigned to an average ofZ-values of the pixels in a matrix MM, the matrix MM including the pixelPx in its upper left corner, and the matrix including 144 dots of 12longitudinal dots (My) and 12 horizontal pixels (Mx). After thesmoothing, deviation of neighboring pixels is smoothed in comparisonwith Z-values obtained in Step S104.

It is possible to employ other sizes of the matrixes MM. The larger asize of the matrix MM is, the larger the number of the pixels effectingan average of the Z-values is. As a result, a difference in Z-valuesamong neighboring pixels is small; thereby a viewer perceives a worlddepicted in a stereographic image as being flatter. On the other hand,the smaller a size of the matrix MM is, the more rough a viewer'sperception of a world depicted in a stereographic image.

In the following, detailed description of the smoothing will be providedwith reference to FIG. 16. It is to be noted that *outinfo is a pointerrepresentative of an address to which Z-values through the smoothing arestored.

In Step S401, a value of a counter CLN which represents a longitudinalposition of a target pixel is reset. Next, it is checked whether a valueof the counter CLN is smaller than the total number of ongitudinalpixels HT (Step S402). If the condition of Step S402 is not satisfied,it is determined that the smoothing has been performed on all the pixelson the screen, and thus the smoothing ends.

In Step S403, it is checked whether a target pixel Px is located 12pixels from the bottom of the screen, or HT-CLN is equal to or largerthan 12. If the condition of Step S403 is satisfied, 12 is assigned to avariable MTh representative of the length of matrix MM (Step S404). Ifthe condition of Step S403 is not satisfied, HT-CLN is assigned to thevariable MTh (Step S405). By this substitution, a size of the matrix MMin a case where a target pixel is located near the bottom of the screenis adjusted.

Next, a counter CWD representative of a horizontal position of thetarget pixel is initialized (Step S406). It is checked whether a currentvalue of the counter CWD is smaller than the total number of horizontalpixels WD (Step S407). If the condition of Step S407 is satisfied, it ischecked whether the target pixel Px is located 12 pixels left or morefrom the right edge of the screen, or WD-CWD is equal to or larger than12 (Step S408).

If the condition of Step S408 is satisfied, 12 is assigned to a variableMTw representative of the width of a matrix MM (Step S409). If thecondition of Step S408 is not satisfied, WD-CWD is assigned to thevariable MTw (Step S410). By this substitution, a size of the matrix MMin a case where the target pixel is located near the right edge of thescreen is adjusted.

Next, with respect to a target pixel specified by values of the countersCLN and CWD, the sum of Z-values (zinfo) of all pixels in the matrix MM(SM1) is obtained, the size of the matrix MM being defined by MTh*MTw(Step S411). SM1/(MTh*MTw), which is an average of Z-values of all thepixels in the matrix MM, is assigned to outinfo (Step S412).

Thus far, the smoothing is performed on a single pixel; thus a currentvalue of the counter CWD is incremented by 1 (Step S413), and theprocessing goes back to Step S407 to perform the smoothing on anothertarget pixel. If the condition of Step S407 is not satisfied, it isdetermined that the smoothing has been performed on all pixels in asingle line. Thus, a current value of the counter CLN is incremented by1 (Step S414) and the processing goes back to Step S402 to perform thesmoothing on another target pixel. Thus, the smoothing is eventuallyperformed on all pixels of the screen.

B-4. Distribution Adjustment

The distribution adjustment (Step S107 of FIG. 7) will now be described.Since Z-values of pixels though the smoothing depend on the average ofZ-values over all the pixels in the image, the distribution of Z-valuesmay be biased. The upper part of FIG. 17 shows an example of a biaseddistribution of Z-values. In this case, a stereographic image generatedusing Z-values without any adjustment will not successfully achievenatural plasticity. In view of the above, the distribution of Z-valuesis revised in this distribution adjustment, as shown in the lower partof FIG. 17.

Referring to FIGS. 18 and 19, the distribution adjustment will bedescribed in detail. *z_image is a pointer representative of an addressin which a Z-value is stored, ave_z is an average value of Z-values ofall pixels in a screen, mix and min_z are the maximum and minimum ofZ-values, respectively, and *adjust_z_image is a pointer representativeof an address in which Z-values through the distribution adjustment isperformed.

Firstly, max_z, min_z, ave_z are calculated (Step S501). Next,max_z−ave_z and min_z−ave_z are assigned to variables max_span andmin_span which are parameters representative of a variance of thedistribution, respectively (Step S502).

Next, it is checked whether the color depth is 16 bits (Step S503). Whenthe color depth is 16 bits, 31 is assigned to a variable maxrepresentative of gradation scale (Step S504). When the color depth is24 bits or 32 bits (Step S503, NO), 255 is assigned to the variable max(Step S505).

Next, a value of the counter CLN is reset to 0 and an address to whichz_image is stored is assigned to a pointer variable *pz_imagerepresentative of a modification value for a Z-value of a target pixel(Step S506). Next, it is checked whether a value of the counter CLN issmaller than the number of total longitudinal pixels HT (Step S507). Ifthe condition of Step S597 is not satisfied, the processing has beenperformed on all pixels; thus, the distribution adjustment ends.

If the condition of Step S507 is satisfied, a value of the counter CWDis reset to 0 (Step S508). In Step S509, it is checked whether a currentvalue of the counter CWD is smaller than the total number of thelongitudinal pixels. If the condition of Step S509 is not satisfied, theprocessing on pixels in a single line is finished; thus, a current valueof the counter CLN is incremented by 1 (Step S510) and the processinggoes back to Step S507 to perform the distribution adjustment on pixelsof the next line.

If the condition of Step S509 is satisfied, it is checked whetherpz_image is larger than ave_z (Step S511). In a case wherepz_image>ave_z, namely the Z-value is larger than the average value,(z−ave_z)/max span*(max-ave_z) is assigned to a variable zrepresentative of a modification value of the pixel (Step S512).

If the condition of Step S511 is not satisfied, it is checked whetherpz_image<ave_z (Step S513). If the condition of Step S513 is satisfied,(z−ave_z)/(min_z−ave_z)*min_span is assigned to the variable z (StepS514). If the condition of Step S514 is not satisfied, or the value of*pz_image is equal to ave_z which is the average of Z-values, 0 isassigned to the variable z (Step S515).

After the variable z is determined at this stage, a value pz_image+z isassigned to *adjust_z_image (Step S516), so as to store a correctedZ-value of the target pixel. Next, values of *pz-image, adjust_z_image,and CLN are incremented by 1 (Step S517), and the processing goes backto Step S509 to perform the distribution adjustment on another pixel.

The local adjustment (Step S108 of FIG. 7) will now be described. Theremay be a case where a further correction is required in addition to theadjustment described above. Specifically, distortions appear in adistribution of Z-values of pixels relating to a single object. Forexample, inside a spherical object Z-values should change gradually andradially from the center to the perimeter. However, a case may arisewhere Z-values of a certain portion of the object are significantlylarge or small for some reason, or a case where irregularities ofZ-values in the distribution may appear.

If a stereographic image is generated using Z-values having a distorteddistribution described above, the order of depth of objects will bedistorted accordingly; thus an image that should be perceived as asingle object appears unnatural to a viewer. In light of this, in thelocal adjustment, Z-values of pixels are corrected on an object basis.

An overview of the local adjustment will be described with reference toFIGS. 21 and 21. Firstly, a range (the number of steps) of Z-valuesobtained in Steps S101 through S107 is corrected (Step S700 of FIG. 21).Next, contour lines HL0 through HL5, each of which is comprised ofpixels of equivalent Z-values which are corrected in Step S700, areobtained (Step S701). Next, objects (e.g. OB1 and OB2 of FIG. 20) areidentified on the basis of the obtained contour lines (Step S702). Ycoordinate of the center of Z-values with respect to the object PPC isidentified. Y coordinates of intersections of the vertical line whichbegins at the point PPC and the contour lines are obtained (Step S703).

Z-values of pixels relating to the object are obtained using thecoordinates calculated above (Steps S704 and S705). Finally, Z-valueswithin a region between the neighboring two lines (hereinafter referredto as a plane of equal altitude) are corrected (Step S706). It is to benoted that when only a single contour line is obtained, the regioninside the line should be regarded as the plane of equal altitude.Detailed processing will be described in the following.

(a) Correction of Gradation Steps

As described above, a minimum unit of displacement of a pixel is apixel, when generating a stereographic image. In other words, it isimpossible to displace a pixel by a distance shorter than a pixel width.As a result, if it is determined that an increment of a Z-value 1corresponds to the minimum displacement distance which is equivalent toa single pixel, the maximum Z-value 255 will result in a displacement ofa pixel by a distance equivalent to a 256 pixel width. However, asignificantly large distance of displacement (e.g. 8 mm on a screen)will cause fatigue for a viewer, and will prevent the viewer fromperceiving the image three-dimensionally. In view of this, CPU 11performs approximation of Z-values obtained in Step S101 through 107 toreplace the Z-values with certain discrete values.

Specifically, CPU 11 divides a Z-value obtained in Step S107 by arounding factor MK, rounds the number of decimal places (rounding up,rounding down, or rounding off to the nearest integer), and multiples itby the factor MK. As a result, the Z-values of pixels are replaced bynumbers which are integer times of the rounding factor MK.

In other words, the rounding factor MK is a number representative of astep size of Z-valued to be descretized. In this embodiment, therounding factor MK is set to a value by dividing the maximum number ofthe Z-values 255 by an object depth Ob specified by a user and roundingit. The object depth Ob is a parameter representative of the number ofsteps available for Z-values. For example, if MK=10, the Z-values can beassigned by 0, 1, 2, 3, . . . , 25, which means that 26 steps areavailable for the Z-values.

(b) Calculation of Contour Lines

Calculation of contour lines will be described in detail referring toFIGS. 22 through 24. *Outline is a pointer representative of an addressin which contour line basic data is stored, and *cord is a pointerrepresentative of an address in which Z-value vectors is stored. Cord isdata having a structure Vinfo. The structure Vinfo includes dataelements including Z-values, npoint (an integer), and *point (coordinatedata).

Firstly, CPU 11 assigns 255 divided by the rounding factor MK to a max,which is a constant, and assigns 0 to a variable Target_z representativeof the number of pixels on which the processing has already beenperformed (Step S710).

Next, it is checked whether a value of the variable Target_z is largerthan the constant value max (Step S711). If the condition of Step S711is not satisfied, calculation of contour lines ends. If the condition ofStep S711 is satisfied, the counter CLN is reset to 0 (Step S712) and itis checked whether a current value of the counter CLN is smaller than HT(Step S713). If the condition of Step S713 is not satisfied, a value ofTarget_z is incremented by 1 (Step S714) and the processing goes back toStep S711. If the condition of Step S713 is satisfied, the counter CWDis reset to 0, and −1 is assigned to the variable P1 and P2, and 0 isassigned to a variable fnd (Step S715).

Next, it is checked whether a value of the counter CWD is smaller thanWD (Step S716). If the condition of Step S716 is satisfied, it ischecked whether a value of *outline is identical to a value of Target_z(Step S717). If the condition of Step S717 is satisfied, it is checkedwhether a value of the variable P1 is larger than 0 (Step S718). If thecondition of Step S718 is not satisfied, the value of the counter CWD isassigned to the variable P1 (Step S719), a value of the counter CWD isincremented by 1 (Step S720), and the processing goes back to Step S716.

If the condition of Step S718 is satisfied, processing is forwarded toStep S720 to perform subsequent processing.

If the condition of Step S717 is not satisfied, it is checked whether acurrent value of the variable P1 is larger than 0 (Step S721). If thecondition of Step S721 is satisfied, a current value of the counter CWDis assigned to the variable P2, a value of the variable fnd isincremented by 1 (Step S722), and the processing goes back to Step S720to perform the subsequent processing. If the condition of Step S716 isnot satisfied, it is checked whether a value of the variable fnd islarger than 0 (Step S723). If the condition of Step S723 is notsatisfied, a current value of the counter CLN is incremented by 1 (StepS724), and the processing goes back to Step S713.

If the condition of Step S723 is satisfied, a search is carried out tofind neighboring pixels of a target pixel, the pixels having Z-valueswhich are different from a Z-value of the target pixel. Specifically,the search is carried out in an order shown in the left part of FIG. 23.At first, a target pixel P1 is compared with the upper right pixel (StepS730). If Z-values of both pixels are the same (Step S730, NO), theZ-value of the target pixel P1 is compared with a Z-value of the pixellocated directly above the pixel P1 (Step S731). Similarly, thecomparison is carried out for the six pixels neighboring the targetpixel (Steps S732 though S735). A schematic diagram of the search orderis shown in the right part of FIG. 23.

If a pixel having a Z-value different from the Z-value of the targetpixel P1 (YES in Steps 730, 731, 732, 733, 734, and 735) is found, it ischecked whether the pixel has previously been found, so as to confirmthat all the pixels around the target parcel P1 have been checked (StepS736). Thus, an edge comprising differing Z-values is extracted. Whenall pixels relating to the edge are checked (Step S736, YES), thecalculation of contour lines ends.

If the condition of Step S736 is not satisfied, x and y coordinates of apixel which were found in the previous processing are assigned to x andy values of point of cord, respectively (Step S737). Next, values pointof cord and npoint are incremented by 1 (Step S738 and S739), the foundpixel is set as the target pixel P1 (Step S740), and the processing goesback to Step S730 to perform the comparison. Thus, edges are found so asto extract a single contour line.

(c) Determination of Object's Center

The determination of an object's center (Step S702 of FIG. 21) will nowbe described with reference to FIGS. 25 though 27. In the determinationof object's center, coordinates of vertexes of a rectanglecircumscribing the inner most contour line relating to a single objectare obtained, a center of the rectangle is regarded as a center ofZ-values of the object. Cent_cord is a pointer representative of avector area for storing the innermost rectangle, center is coordinatesof a center to be obtained, and *side_cord is a pointer representativeof a vector area for storing the rectangle other than the innermost one.

At first, CPU 11 selects a contour line (Step S750). Next, variablesg_min_x, g_man_x, g_min_y, g_man_y, that represent the minimum and themaximum values of x and y coordinates of pixels on the contour line areinitialized (Step S751). In FIG. 25, for simplicity, the initial valueof g_min_x is shown as 999999, but other values which are larger than WDcan be used instead. This also applies to g_min_y. In Step S752 a valueof cord is assigned to cordp.

Next, it is checked whether a value of npoint of cord is larger than 0(Step S753). If the condition of Step S753 is not satisfied, it ischecked whether processing is performed on all contour lines (StepS754). If the condition of Step S754 is not satisfied, the processinggoes back to Step S750 to perform processing on another contour line. Ifthe condition of Step S754 is satisfied, the processing ends. If thecondition of Step S753 is satisfied, variables are initialized.Specifically, a value of the counter is reset to 0 (Step S755), 999999is assigned to min_x, 0 is assigned to max_x, 999999 is assigned tomin_y, 0 is assigned to max_y (Step S756), and a value of npoint of cordis assigned to a variable PNT.

Next, it is checked whether a current value of the counter CNT issmaller than the value of npoint of cord, namely whether the processinghas been performed on all pixels relating to a single contour line ischecked (Step S757). If the processing is not finished (Step S757, YES),it is checked whether an x value of the variable PNT is smaller thanmin_x (Step S758). If the condition of Step S758 is satisfied, thehorizontal minimum position is assigned to P1. Specifically, an x valueof the PNT is assigned to p1.x and a y value of the PNT is assigned top1.y, and pi.x is assigned to min_x (Step S759). If the condition ofStep S758 is not satisfied, Step 759 is omitted.

Next, it is checked whether the x value of PNT is larger than max_x(Step S760). If the condition of Step S760 is satisfied, the maximumhorizontal position is assigned to p2. Specifically, the x value of PNTis assigned to p2.x and the y-value of PNT is assigned to p2.y, and p2.xis assigned to max_x (Step S761). If the condition of Step S760 is notsatisfied, Step S761 is omitted.

Next, it is checked whether the y value of the variable PNT is smallerthan min_y (Step S762). If the condition of Step S762 is satisfied, theminimum vertical position is assigned to p3. Specifically, the x valueof PNT is assigned to p3.x, the y value of PNT is assigned to p3.y, andp3.y is assigned to min_y (Step S763). If the condition of Step S762 isnot satisfied, Step S763 is omitted.

Next, it is checked whether the y-value of the variable PNT is smallerthan max_y (Step S764). If the condition of Step S764 is satisfied, themaximum vertical position is assigned to p4. Specifically, an x value ofPNT is assigned to p4.x, the y value of PNT is assigned to p4.y, andp4.y is assigned to max_y (Step S765). If the condition of Step S672 isnot satisfied, Step S765 is omitted. Next, a value of the counter PNT isincremented by 1 (Step S766), and the processing goes back to Step S757.

When the vertexes of a circumscribing rectangle (Step S757, NO) iscalculated, it is checked whether p1.x is smaller than g_min_x todetermine whether the rectangle is the innermost one. If the rectangleis disposed innermost (Step S767, YES), cordp is stored in cent cord(Step S768) and p1.x is assigned to g_min_x (Step S769). Next, anintersection of a line segment starting at the vertex of the innermostrectangle (p1.x, p3.y) and ending at a point (p2.x, p4.y), and a linesegment starting at a point (p2.x, and p3.y) and ending at a point(p1.x, p4.y), is calculated. X and y coordinates of the intersection arestored in center.x and center.y, respectively (Step S770). If therectangle is not the innermost one (Step S767, NO), Steps S768 through770 are skipped.

Next, it is checked whether p2.x is larger than g_max_x (Step S771). Ifp2.x>g_max_x (Step S711,YES), cordp is stored in side_cord and p2.x isstored in g_max_x (Steps S772 and S773). If p2.x is smaller than g_max_x(Step S771, NO), Steps S772 and 773 are omitted. Next, a value of cordpis incremented by 1 (Step S774) and the processing goes back to StepS753 to perform similar processing on another contour line.

(d) Determination of Intersections

The intersection determination (Step S703 of FIG. 21) will be describedwith reference to FIG. 28. It is to be noted that p[] is an array forstoring coordinates of an intersection. At first, a contour line isselected (Step S780), cord is assigned to cordp, and a variable find isset to 0 (Step S781). Next, it is checked whether npoint of cord islarger than 0 (Step S782). If the condition of Step S782 is notsatisfied, it is checked whether the calculation has been performed onall contour lines (Step S783). If the condition of Step S783 is notsatisfied, the processing goes back to Step S780 to perform theintersection determination on another contour line. If the condition ofStep S783 is satisfied, the intersection determination ends.

If the condition of Step S782 is satisfied, npoint of cordp is assignedto the variable PNT and a value of the counter CNT is reset to 0 (StepS784). Next, it is checked whether a value of the counter CNT is smallerthan the npoint of cord (Step S785). If the condition of Step S785 issatisfied, it is checked whether PNT.x is identical to center.x (StepS786).

If the condition of Step S786 is satisfied, it is checked whether PNT.yis larger than center.y (Step S787). If the condition of Step S787 issatisfied, an intersection of a vertical line starting at the center andthe contour line is determined. Specifically, PNT.x is assigned top[find] and PNT.y is assigned to p[find] (Step S788).

Next, a value of the variable find is incremented by 1 (Step S789), and−1 is assigned to p[find].x and p[find].y (Step S790). Next, a value ofcordp is incremented by 1 (Step S791) and the processing goes back toStep S782.

If the condition of Step S786 or 787 is not satisfied, values of CNT andPNT are incremented by 1 (Step S792), and the processing goes back toStep S785. If the condition of Step S785 is not satisfied, theprocessing goes back to Step S791.

(e) Inplane Z-Value Calculation

The inplane Z-value calculation (Step S704 of FIG. 21) will be describedwith reference to FIG. 29. In this calculation, Z-values within a planeof equivalent level are revised using coordinates of the center of anobject and the intersections which are obtained in the previousprocessing. A constant value dist represents a distance between aposition of a camera and a screen assuming that the image displayed onthe screen is shot by a camera, a constant value of focus represents afocal length of the camera, and new_z[] represents an array for storinga Z-value calculated in the inplane Z-value calculation.

At first, CPU 11 selects a contour line (Step S795) and resets a valueof the counter CNT to 0 (Step S796). Next, HT subtracted by center.y isassigned to cent_h and dist/focus*cent_h is assigned to object_h, whichis representative of a depth of the object (Step S797). Next, it ischecked whether a value of P[CNT].x is larger than 0 (Step S798). If thecondition of Step S798 is satisfied,object_h*focus/(p[CNT].y-center.y)-dist is new_z[CNT] is assigned tonew_z[CNT] (Step S799), so as to revise a Z-value of a pixel within aplane of equivalent level, so that the Z-value is proportional to a ycomponent of a distance between the center of Z-values with respect tothe object and the pixel.

Next, a value of the counter CNT is incremented by 1 (Step S800) andthen the processing goes back to Step S798. If the condition of StepS798 is not satisfied, it is checked whether the inplane Z-valuecalculation has been performed on all the contour lines (Step S801). Ifthe condition of Step S801 is not satisfied, the processing goes back toStep S795 to perform the inplane calculation on another contour line. Ifthe condition of Step S801 is satisfied, the inplane calculation ends.

(f) Calculation of Inplane Modification Z-Value

In the calculation of inplane modification of a Z-value, a Z-valuewithin a plane of equivalent level is corrected using a Z-value obtainedin Step S704 of FIG. 21. In the following, detailed description on thecalculation of inplane modification Z-value will be provided withreference to FIG. 30. It is to be noted that adjust_outline[] is anarray for storing a difference of Z-values before the correction andafter the correction.

At first, an intersection obtained in the previous processing isselected (Step S805) and 1 is assigned to a value of the counter CNT(Step S806). Next, it is checked whether p[CNT].x is larger than 0 (StepS807). If the condition of Step S806 is not satisfied, it is determinedthat the processing is finished for a single intersection, and it ischecked whether the processing has been performed on all theintersections (Step S808). If the condition of Step S808 is notsatisfied, the processing goes back to Step S805 to perform theprocessing on another intersection. If the condition of Step S808 issatisfied, the calculation of inplane modification of Z-value ends.

If the condition of Step S807 is satisfied, a Z-value of a pixelidentified by a coordinate (p[CNT-1].x, p[CNT-1].y-1) is assigned to avariable oz1 (Step S809) and a Z-value of a pixel identified by acoordinate (p[CNT].x, p[CNT].y-1) is assigned to a variable oz2 (StepS810). Next, oz2-oz1 is assigned to a variable sub1 (Step S811) andnew_z[CNT]-new_z[CNT-1], which is an increment by which a Z-value isadjusted is assigned to a variable sub2 (Step S812).

Next, it is checked whether signs of sub1 and sub2 are the same (StepS813). If the signs are the same (Step S813, YES), 0 is assigned toadjust_outline[CNT] (Step S814). If the signs are different (Step S813,NO), sub2-sub1 is assigned to adjust_outline[CNT] (Step S815). Next, thecounter CNT is incremented by 1 (Step S816) and then the processing goesback to Step S807.

(g) Object-Basis Z-Value Adjustment

In this adjustment, Z-values of pixels in an object are adjusted usinginplane modification Z-value obtained in Step S705. Detailed descriptionof the object-basis Z-value adjustment (Step S109 of FIG. 7) will beprovided in the following with reference to FIG. 31. It is to be notedthat *adjust_Z_image is a pointer representative of an address in whichadjusted Z-values are stored.

At first, the counter CLN is reset to 0 (Step S901) and it is checkedwhether a current value of the counter CLN is smaller than HT (StepS902). If the condition of Step S902 is not satisfied, the processingends. If the condition of Step S902 is satisfied, 0 is assigned to thecounter CWD and it is checked whether CWD is smaller than WD (StepS904). If the condition of Step S904 is satisfied, adjust_z_image of thetarget pixel is obtained (Step S905), and the z_image is updated byadding adjust_z_image to the z-image (Step S906). Thus far, update of aZ-value of a single pixel is finished.

Next, a value of CNT is incremented by 1 (Step S907) and the processinggoes back to Step S904 to perform the processing on another pixel. Ifthe condition of Step S904 is not satisfied, a value of CWD isincremented by 1 (Step S908) and the processing goes back to Step S902to perform processing on pixels on another line. Thus, Z-values of allthe pixels are adjusted.

<B-6. Displacement of Pixels>

In this section, parallax (the amount of displacement ) is determined onthe basis of a Z-value obtained thus far, and a parameter specified by auser, and a pixel is moved horizontally according to the amount ofdisplacement . Overall processing of displacement of pixels (Step S110of FIG. 7) is shown in FIG. 32. It is to be noted that *image3d is apointer representative of an address in which stereographic image datais stored.

At first, the counter CLN is reset to 0 (Step S1001) and it is checkedwhether a current value of CLN is smaller than HT (Step S1002). If thecondition of Step S1002 is not satisfied, the processing ends. If thecondition of Step S1002 is satisfied, the counter CWD is reset to 0(Step S1003).

Next, it is checked whether a current value of CWD is smaller than WD(Step S1004). If the condition of Step S1004 is satisfied, a range ofZ-values corresponding to the respective outinfo are adjusted (StepS1005). A Z-value obtained in Step S 1005 is hereinafter referred to asa ZZ. Calculation of ZZ will de described with reference to FIG. 33.

At first, values available for a Z-value are adjusted using the roundingfactor MK (Steps S1020 and S1021). This adjustment is similar to theprocessing of Step S700. A Z-value obtained in Steps S1020 and S1021 ishereinafter referred to as a ZZ.

Next, it is checked whether ZZ is larger than an upper limit (Step S1022). The upper limit is input by a user in the input box IB3 of thesetting dialogue panel. If ZZ is larger than the upper limit (StepS1022, YES), ZZ is replaced by the upper limit (Step S1023). Similarly,it is checked whether ZZ is smaller than a lower limit (Step S1024). Thelower limit is input by a user in the input box IB4 of the settingdialogue panel. If ZZ is smaller than the lower limit (Step S1024, YES),ZZ is replaced by the lower limit.

By the foregoing process, Z values lying beyond the upper and the lowerlimits are caused to fall within the predetermined range. Thisprocessing is employed in view of the fact that the number of pixelswith large Z-values and small Z-values is small in comparison to thetotal number of the pixels. When a range of available Z-values areadjusted insofar as the pixel crossing phenomenon does not appear,quality of a generated stereographic image is improved.

Referring again to FIG. 32, parallax SS is calculated using ZZ obtainedin the previous steps. Calculation of SS will now be described withreference to FIG. 34. In the reverse determination of Step S105, it ischecked whether reverse of a Z-value is effected (Step S1030). If thereverse is effected (Step S1030, YES), 256-ZZ is assigned to ZZ (StepS1031). Next, ZZ added by an interface depth is assigned to SS (StepS1032). The interface depth is input via Input Box IB1 of the settingdialogue panel. If the reverse is not effected (Step S1032, NO), theprocessing proceeds to Step S1032.

Referring again to FIG. 32, when the parallax SS is calculated, a parityof a current value of the counter CLN is checked (Step S1007). If thevalue of the counter CLN is even (Step S1007, YES), CN+SS is assigned toMV which represents the amount of displacing a pixel (Step S1008). Ifthe value of the counter CLN is odd, CN−SS is assigned to MV (StepS1009). Thus, a pixel is shifted in opposite directions to generateimages for the right and left eyes. Next, pixel data of a positionspecified by src+MV in the original 2D data is stored in image3d (StepS1010). Thus far, the processing on a single pixel is finished.

Next, a value of image3d, outinfo, and the counter CWD are incrementedby 1 (Step S1001), and the processing goes back Step S1004 to performthe processing on another pixel. The processing is repeatedly performedon the pixels on a single line. After the processing is performed on allthe pixels in a line (Step S1004, NO), src is set to the beginning ofthe next line (Step S1012), a current value of CLN is incremented by 1(Step S1013), and the processing goes back to Step S1002 to perform theprocessing on the next line. When the processing has been performed onall the lines (Step S1002, NO), the processing ends.

As described thus far, Z values are adjusted on an object basis thatshould be recognized as a single object, thus an order of depth relatingto pixels within the object is maintained. As a result, a generatedstereographic image successfully gives to a viewer a realisticimpression.

In this embodiment, since the adjustments described above are performedon a Z-value, it is possible to generate a stereographic image with ahigh quality. In addition, since parameters are input by a user via thesetting dialogue panel used for generating a stereographic image, it isalso possible to generate a stereographic image in accordance with auser's preferences.

Although in this embodiment the adjustment of Steps S103 through S109shown in FIG. 7 are sequentially carried out, it is not necessary tocarry out all of Steps S103 through S109. Needless to say, it ispossible to select a preferred adjustment(s) to be effected, from amongthe adjustments of Steps S103 through S 109.

<C. Modifications of the First Embodiment>

<1. Enhancements>

It is possible to give to a stereographic image an effect which enhancesplasticity of a particular object(s) among objects displayed in ascreen. The enhancement is effective especially for a case in which anobject moves in a direction depth. FIG. 35 is a schematic diagramillustrating the enhancement. Specifically, the enhancement is carriedout by addition of a predetermined value to Z-values in planes of equalaltitude included in an object OBa, as shown in FIG. 35.

FIG. 36 shows an example of a processing flow of the enhancementperformed when 3D mode is effected. The processing shown in FIG. 35differs from the first embodiment in that it performs the enhancement(Step S1110) before the displacement of pixels (Step S111). Steps S1101through S1119 are the same as the respective steps of the firstembodiment, and description thereof is therefore omitted.

The enhancement (Step S1110) will be described referring to FIG. 37. Itis to be noted that offset_z is a parameter indicating a degree ofenhancing an object. At first, a terminate check is performed in StepsS1120 through S1222. Specifically, the counter CLN is reset to 0 (StepS1120), and it is checked whether a current value of the counter issmaller than HT (Step S1121). If the condition of Step S1121 is notsatisfied, the enhancement ends. If the condition of Step S1121 issatisfied, a value of CLN is reset to 0 (Step S1122).

Next, it is checked whether a current value of the counter CWD issmaller than WD (Step S1123). If the condition of Step S1123 issatisfied, a half line is drawn starting at a coordinate (CWD, CLN) andending at the top of the upper edge of the image (Step S1124), and thenumber of intersections of the half line and the contour lines isobtained (Step S1125).

Next, a parity of the number of intersections is checked (Step S1126).If the number of the intersections is odd (Step S1126, YES), it isdetermined that the coordinate (CWD, CLN) lies inside a correspondingrectangle. Next, z_image+CLN*WD+CWD is assigned to pz_image (StepS1128), and a value of *pz_image+CLN*WD+CWD *to *pz_image (Step S1129).If the number of intersections is even (Step S1126, NO), Steps 1127through S1129 are omitted. Thus far, the enhancement is performed on asingle pixel. Next, a value of the counter CWD is incremented by 1, andthe processing goes back to Step S1123 to perform the enhancement onanother pixel. This processing ends when the enhancement is performed onall the pixels.

<2. Display Scheme>

In the above embodiment, so called Odd-even scheme for displaying astereographic image is employed, in which images for the left eye aredisplayed in the even-numbered lines of a screen of Display 21 andimages for the right eye are displayed in the odd-numbered lines of ascreen of Display 21. However, other display schemes can be employed inthe present invention. For example, it is possible to display images forthe right eye in the even-numbered lines of a screen of Display 21 andimages for the left eye in the odd-numbered lines of a screen of Display21.

Further, it is possible to employ a so called Up-down scheme, as shownin FIG. 38, in which data of the even-numbered lines of an image for theleft eye are read and stored in the upper part of a frame buffer, anddata of the odd-numbered lines of an image for the right eye are readand stored in the lower part of the buffer. This scheme is employedtypically in a projector for displaying a stereographic image or thelike. Needless to say, it is possible to read data of the even-numberedlines of an image for the right eye and store it in the upper part ofthe buffer, and read data of the odd-numbered lines of an image for theleft eye and store it in the lower part of the buffer.

Still further, it is possible to employ a so called Left-right schemeshown in FIG. 39, in which data of the even-numbered rows for the lefteye is read and stored in the left part of a frame buffer, and data ofthe odd-numbered rows for the right eye is read and stored in the rightpart of the buffer. This scheme is typically employed in a CRT fordisplaying a stereographic image. Needles to say, it is possible tostore data of the even-numbered rows for the right eye in the left partof the buffer and to store data of the odd-numbered rows for the lefteye in the right part of the buffer.

Still further, it is possible to employ a so called two-frame scheme, asshown in FIG. 40, in which an image for the left eye is stored ineven-numbered frame buffers and an image for the right eye is stored inodd-numbered frame buffers. This scheme is typically employed in aprojector for displaying a stereographic image or the like. Needless tosay, it is possible to store image data for the right eye ineven-numbered frame buffers, and to store image data for the left eye inodd-numbered frame buffers.

Still further, it is possible to employ a so called Odd-even fileseparation scheme, as shown in FIG. 41, in which image data for the lefteye and image data for the right eye are stored in files separately.This scheme is typically employed in a projector for displaying astereographic image.

<3. Data Storage Area>

It is possible to store images for the right and the left eyes indifferent storage areas. In this case, the displacement of pixels (StepS110 of FIG. 7) is carried out according to a flow shown in FIG. 42. Thedisplacement of pixels according to the modifications of the firstembodiment differs from the displacement of pixels according to thefirst embodiment itself in performing Steps S1057 through S1060 of FIG.42, instead of Steps S1007 through S1011 of FIG. 32. Specifically, imagedata for the right and the left eyes is respectively stored in datastorage areas which are identified by Rimage3d and Limage3d for the leftand the right eyes.

More specifically, when parallax SS is obtained in Step S1056, CN+SS isassigned to MV1 and CN−SS is assigned to MV2 (Step S1057). Next, data ofa pixel at a position identified by src+MV1 which is included inoriginal 2D image data is stored in Limage3d (Step S1058), and data of apixel at a position specified by src+MV2 is stored in Rimage3d (StepS1059). Next, values of Limage3d, Rimage3d, outinfo, and CWD areincremented by 1 (Step S1060), and the processing goes back to StepS1054 to perform the enhancement on another pixel.

<4. Data Type>

In the above embodiment image data is data of a 2D color image; however,the present invention can be adapted to data of a 2D monochrome multigradation image such as a monochrome film. Also, in the above embodimentoriginal data to be processed is data for a single frame of movingimages; however, it is possible to process data of a freeze framepicture in the present invention.

<5. Calculation of Z-Value>

In the above embodiment R, G, and B values of an original image aremultiplied by the coefficients 0.59, 0.30, 0.11, respectively, togenerate a Z value to be processed by the subsequent adjustments;however, other coefficients can be used. The coefficients can bemodified to obtain the most natural plasticity from the generated 3Dimage, or can be modified according to a user's preferences. Also, it ispossible to describe a color of a pixel on the basis of brightness,saturation or hue, instead of RGB. Also, it is possible to use ARGB, inwhich A indicates an alpha component of a pixel, and the like.

<6. Identification of Objects>

In the above embodiment, contour lines with regard to Z-values areobtained, an object is identified on the basis of the contour lines, andthen Z-values of pixels relating to the object are corrected. However,other methods for identifying an object can be used in the presentinvention. For example, edges are found based on information onbrightness, saturation, or hue of pixels, so as to determine that anarea enclosed by the edges corresponds to an area of a single object.

SECOND EMBODIMENT

<A. Configuration of Apparatus>

The second embodiment of the present invention will now be described.Like numerals are assigned to like elements of the first embodiment. Thesecond embodiment differs from the first embodiment in that a Mobileterminal 1A is used instead of 3D image generation apparatus 1. FIG. 43shows a functional configuration of Mobile terminal 1A and FIG. 44 showsan external view of Mobile terminal 1A. As shown in FIG. 44, Mobileterminal 1A is, for example, a mobile phone. The following descriptionmainly describes differences between the first and the secondembodiments.

FIG. 43 shows a radio communication unit 27, which includes an antennaAT, for connecting Mobile terminal to a mobile communications network totransmit and receive data such as moving images data. The networkincludes a communications unit 28, which includes a speaker unit and amicrophone unit, for carrying out a voice communications. A Display 21 ahas a function of displaying 2D mode images and 3D images alternatively,and includes a display panel for exciting stereopsis when an image isviewed by the naked eye. A display control unit 22 a controls Displayunit 21 a. An operations unit 30 is an input device such as a ten keypad or a cursor key, via which a user inputs directions to Mobileterminal 1A.

The principle of exciting stereopsis when an image is viewed by thenaked eye will be briefly described with reference to FIGS. 45 and 46.As shown in FIG. 45, Display 21 a includes a backlight 211, liquidcrystal panel 212, and liquid crystal panel 213 on which parallaxbarriers are disposed. Striped formed images for the left eye L1 throughL5 and striped formed images for the right eye R1 through R5 in thepanel are displayed at a spacing equivalent to a slit width,respectively, so that the images L1 through L5 reach only the left eyeof a viewer and the images R1 through R5 reach only the right eye of theviewer when the viewer is at a predetermined distance from the screen.In this case, if images are generated so that a difference of parallaxof the images L1 through L5 and the images R1 through R5 is a constant,the images excite stereopsis when viewed by the naked eye.

Referring again to FIG. 44, a ten key pad TK is disposed in the lowerbody of Mobile terminal 1A, and above TK are disposed function keys anda switch DK used for switching modes between a normal mode (2D mode) anda stereograph mode (3D mode). A screen of Display 21 a is disposed aboveFK and DK. Antenna AT is disposed above the screen. Holes 28 a and 28 bfor a speaker unit included in Radio Communication unit 28 and amicrophone unit, respectively, are disposed at suitable positions ofMobile terminal 1A.

<2. Operations>

<A. Display of Static Image>

FIG. 47 shows an example of a processing flow of displaying a staticimage performed by Mobile terminal 1A, the image being stored in Mobileterminal 1A. At first, after a user switches on Mobile terminal 1Aperforms certain operations to select an image (Step S2000), CPU 11enables the normal mode (2D mode) (Step S2001). Next, CPU 11 outputsimage data to a display control unit 22 a (Step S2002). As a result, theuser is able to view the static image in the usual manner.

When the user inputs an instruction to stop reproduction of the image(Step S2003, NO and Step S2004, YES), CPU 11 terminates displaying theimage (Step S2005).

When Switch DK is operated (Step S2003, YES), CPU 11 enables thestereograph mode (Step S2006) and starts generating data of astereographic image (Step S2007). CPU 11 outputs the generated data toDisplay control unit 22 a (Step S2008). As a result, the user is able toview the stereographic image. A method for generating data of astereographic image is the same as the method employed in the firstembodiment, and description thereof is therefore omitted.

When the user inputs an instruction to stop reproduction of the image(Step S2009, NO and Step S2110, YES), CPU 11 terminates displaying theimage (Step S2111). When the user operates Switch DK once again (StepS2009, YES), the processing goes back to Step S2001 and the normal mode(2D mode) is effected.

<B. Display of Moving Images>

FIG. 48 shows an example of a processing flow for downloading movingimages via a mobile communication network and displaying them in Mobileterminal 1A. When Mobile terminal 1A is turned on, CPU 11 enables normalmode (2D mode) (Step S3001). When a user carries out predeterminedoperations, data of moving images including a plurality of frames isreceived (Step S3002). CPU 11 stores the received data in RAM 13 andoutputs data of the first of the frames to Display control unit 22 a(Step S3003).

Next, CPU 11 determines whether the data includes subsequent frames(Step S3004). In a case where the data does not include subsequentframes (Step S3004, NO), or in a case where a user inputs an instructionto terminate reproduction of the images although the data includessubsequent frames (Step S3007, YES), CPU 11 terminates reproduction ofthe images. If the data includes subsequent frames (Step S3004, YES),CPU 11 reads 20 data of another frame from RAM 13 and outputs it toDisplay control unit 22 a.

By the above process, image data is read and output to the display frameby frame, until a user inputs an instruction to terminate reproductionof the images (Step S3006, NO and Step S3007, NO). When a user inputs aninstruction to terminate reproduction of the images (Step S3006, NO andStep S3007, YES), reproduction of the images is terminated (Step S3005).

Thus far, a user of Mobile terminal 1A is able to view moving images ina normal mode in a default setting.

When a user operates Switch DK (Step S3006, YES), CPU 11 enablesstereograph mode (Step S3008). Next, CPU 11 read from RAM 13 image dataof a frame subsequent to the frame (Step S3009). CPU 11 performs theprocessing on the read data so as to generate a stereographic imagedescribed above (Step S3010), and outputs processed data to Displaycontrol unit 22 a (Step S3011). By this process, a user is able to viewa moving stereographic image.

The stereographic moving images are displayed frame-by frame until atleast of the following conditions is satisfied: there is no subsequentframe (Step S3012, NO); a user operates Switch DK (Step S3013, YES); anda user inputs a direction of terminating reproduction of the images(Step S3014, YES).

When a user operates Switch DK (Step S3013, YES), the processing goesback to Step S3001, and CPU 11 enables normal mode (2D mode) and outputsimage data of the next frame to Display control unit 22 a.

In this embodiment, when a user operates Switch DK, images (including astatic image and moving images) displayed in Display 21 a areimmediately switched between 2D images and 3D images. Thus, a user isable to switch images between 2D images and 3D images while viewing theimages. The capability of switching provides unprecedented enjoymentwith a viewer. For example, it is possible that a user generally watchesmoving images in 2D mode and switches mode to 3D mode to enjoy afavorite scene.

Although in this embodiment data of moving images is assumed to bedownloaded in toto, methods for receiving and reproducing moving imagesare not restricted to those described above. For example, live streamingcan be employed in which Mobile terminal 1A receives data of images oneafter another.

THIRD EMBODIMENT

FIG. 49 shows a functional configuration of a 3D generating apparatus 1Bof the third embodiment. 3D generating apparatus 1B differs from 3Dgenerating apparatus 1 of the first embodiment in having a Displaycontrol unit 22 b, the first data buffer 31, and the second data buffer32 instead of Display control unit 22, and having a Display 21 b insteadof Display 21. In this embodiment, a scheme for exciting stereopsis in auser wearing Glasses 34 in which liquid crystal shutters are disposed isused.

Similarly to the second embodiment, in 3D image generating apparatus 1Bstereogram mode and normal mode can be selected. In 3D mode, Displaycontrol unit 22 b stores image data for the left and the right eyes inFirst data buffer 31 and Second data buffer 32, respectively. Displaycontrol unit 22 b controls a Switch 33 to select First data buffer andSecond data buffer alternately. Specifically, a screen is refreshed at apredetermined time interval on Display 21 b, and Display control unit 22b switches Switch 33 in synchronization with the refresh interval. Inaddition, Display control unit 22 b controls switching the shutters ofGlasses 34 in synchronization with the refresh interval.

More specifically, when Switch 33 selects First data buffer 31, shuttersdisposed on the glass for the left eye are turned on (i.e. transparentstate) and shutters disposed on the glass for the right eye are turnedoff (i.e. cutoff state). On the other hand, when Switch 33 selectsSecond data buffer 31, shutters disposed on the glass for the right eyeare turned on (i.e. transparent state) and shutters disposed on theglass for the left eye are turned off (i.e. cutoff state). By thisprocess, images for the left eye reach only the left eye and images forthe right eye reach only the right eye, thereby stereopsis is excited.When 2D mode is effected, Display control unit 22 b stores image data inFirst data buffer and controls Switch 33 to select First data buffer 31.

FIG. 50 shows an example of a processing flow for displaying movingimages which have been received using 3D generating apparatus 1B. Theoutline of the processing flow is the same as the flow of the secondembodiment described with reference to FIG. 48, except that Steps S3003b and S3011 b are performed instead of Steps S3003 and S3011,respectively.

More specifically, in 2D mode CPU 11 extracts image data of a singleframe from the data stored in RAM 13, stores it in First data buffer 31,and then selects First data buffer 31 of Switch 33 (Step S3003 b). As aresult, a user is able to view 2D moving images in a default setting.

When 3D mode is effected, Display control unit 22 b stores image datafor the left eye generated in Step S3010 in First data buffer 31, andstores image data for the right eye in Second data buffer 32 (Step S3011b). Next, Display control unit 22 b switches Switch 33 between First andSecond data buffers synchronous with the refresh interval of Displayunit 21 b. As a result, a user can successfully view the image of theframe three-dimensionally. Similar processing is performed on otherframes one after another, thereby providing a user with stereopsis forthe moving images.

<Applications>

It is possible to apply 3D generating apparatuses of the presentinvention to simulators used for training in controlling a car, train,airplane, and the like. Specifically, displaying stereographic images ina screen of the simulators enables a trainee to perceive displayedimages more realistically. FIG. 51 shows an example of a Drivingsimulator 40 in which 3D image generating apparatus 1B is incorporated.

As shown in FIG. 51, an Operations unit 41 includes a control devicesuch as a steering wheel. An instruction input by a user is supplied toa Simulation processing unit 42. Simulation processing unit 42 performsprocessing on the basis of the input instruction by Operations unit 41,to generate image data to be displayed on the screen. The image data isoutput to a Frame buffer 43.

When 2D mode (normal mode) is effected, 2D images are displayed in 3Dimage generating apparatus 1B. When 3D mode is effected, image data of asingle frame is extracted from Frame buffer 43 and data of astereographic image is generated. As a result, a stereographic image isdisplayed in the screen.

A method for generating a 3D image according to the present inventioncan be implemented by LSI (large scale integrated circuit).Stereographic images can be displayed in various kinds of electronicdevices incorporating the LSI. For example, a method for generating a 3Dimage based on the present invention can be applied to a communicationterminal, digital camera, TV receiver, or storage display, or to medicalappliances such as endoscopes.

Although preferred embodiments of the present invention have beendescribed thus far, it will be understood that they are not intended tolimit the invention to these embodiments. On the contrary, the inventionis intended to cover alternatives, modifications, and equivalents, whichmay be included within the scope of the invention.

1. A method for a stereographic image generation apparatus forgenerating a stereographic image, the method comprising: a calculatingstep of calculating Z-values of each pixel based on only image data forthat pixel, the pixels forming an image, each Z-value being assigned toa pixel, and each Z-value representing a depth of an objectcorresponding to the pixel; an adjusting step of adjusting a Z-value ofa target pixel obtained in said calculating step using a Z-value of atleast one individual pixel other than the target pixel; and a generatingstep of determining an amount of displacement of the target pixel on thebasis of an adjusted Z-value of the target pixel, and displacing thetarget pixel horizontally by the determined amount, to generate imagesfor the right and the left eyes, wherein: in said calculating step theZ-value of each target pixel is obtained by adding predetermined weightsto color components of image data of the target pixel, and in saidadjusting step: the Z-values of each pixel are adjusted so that a singlestep available for a Z-value of a pixel corresponding to an objectlocated backward in an original image express deeper depth than a singlestep available for a Z-value of a pixel corresponding to an objectlocated forward in the original image; an average of Z-values of allpixels within an area which includes the target pixel is obtained; theZ-value of the target pixel is replaced by the obtained average; a stepsize of quantization of the Z-value is determined based on a value of aparameter specified by a user; and either an upper limit or a lowerlimit of the calculated Z-value is determined based on the value of theparameter specified by the user.
 2. The method of claim 1, wherein theweights are determined based on the ratio of cone cells sensitive of R,G, and B, respectively, which cones exist in a retina of a human eye. 3.The method of claim 1, wherein in said adjusting step: tendency ofZ-values of pixels in the image is analyzed by comparing a Z-value of apixel within an area with a Z-value of a pixel within another area; andwhen a result of the analysis agrees with a predetermined condition, aquantitative relation between the amount of displacement of the targetpixel and the Z-value of the target pixel is reversed in said generatingstep.
 4. The method of claim 1 wherein in said adjusting step: adistribution of the Z-values of all pixels in the image and an averageof all pixels in the image are obtained; and deviation of thedistribution is corrected using the obtained average.
 5. The method ofclaim 1, wherein in said adjusting step: at least one object in theimage represented by the image data is identified referring to Z-valuesof pixels calculated in said calculating step; and a Z-value of thetarget pixel is adjusted on the basis of a Z-value of a pixel locatedwithin an area corresponding to the identified object.
 6. The method ofclaim 1, further comprising a step of obtaining moving images comprisedof a plurality of images, and wherein a stereographic image is generatedfrom each image, to generate stereographic images corresponding to themoving images in real time.
 7. A stereographic image generatingapparatus for generating a stereographic image comprising: a calculatingmeans for calculating Z-values of each pixel based on only image datafor that pixel, the pixels forming an image, each Z-value being assignedto a pixel, and each Z-value representing a depth of an objectcorresponding to the pixel; an adjusting means for adjusting a Z-valueof a target pixel obtained in said calculating means using a Z-value ofat least one individual pixel other than the target pixel; and agenerating means for determining an amount of displacement of the targetpixel on the basis of an adjusted Z-value of the target pixel, anddisplacing the target pixel horizontally by the determined amount, togenerate images for the right and the left eyes, wherein: saidcalculating means obtain the Z-value of each target pixel by addingpredetermined weights to color components of image data of the targetpixel, and said adjusting means: adjust the Z-values of each pixel sothat a single step available for a Z-value of a pixel corresponding toan object located backward in an original image express deeper depththan a single step available for a Z-value of a pixel corresponding toan object located forward in the original image; obtain an average ofZ-values of all pixels within an area which includes the target pixel;replace the Z-value of the target pixel by the obtained average;determine a step size of quantization of the Z-value based on a value ofa parameter specified by a user; and determine either an upper limit ora lower limit of the calculated Z-value based on the value of theparameter specified by the user.
 8. The apparatus of claim 7, furthercomprising an obtaining means for obtaining from the user the parametersused in said adjusting means.
 9. The apparatus of claim 8, furthercomprising: storing means for storing image data for the right and theleft eyes; and displaying means for displaying an image represented bythe image data stored in said storing means in compliance with apredetermined scheme.
 10. A computer program product embodied on acomputer readable medium, which when executed by a computer causes thecomputer to function as: a calculating means for calculating Z-values ofeach pixel based on only image data for that pixel, the pixels formingan image, each Z-value being assigned to a pixel, and each Z-valuerepresenting a depth of an object corresponding to the pixel; anadjusting means for adjusting a Z-value of a target pixel obtained insaid calculating means using a Z-value of at least one individual pixelother than the target pixel; and a generating means for determining anamount of displacement of a target pixel on the basis of an adjustedZ-value of the target pixel, and displacing the target pixelhorizontally by the determined amount, to generate images for the rightand the left eyes, wherein: said calculating means obtain the Z-value ofeach target pixel by adding predetermined weights to color components ofimage data of the target pixel, and said adjusting means: adjust theZ-values of each pixel so that a single step available for a Z-value ofa pixel corresponding to an object located backward in an original imageexpress deeper depth than a single step available for a Z-value of apixel corresponding to an object located forward in the original image;obtain an average of Z-values of all pixels within an area whichincludes the target pixel; replace the Z-value of the target pixel bythe obtained average; determine a step size of quantization of theZ-value based on a value of a parameter specified by a user; anddetermine either an upper limit or a lower limit of the calculatedZ-value based on the value of the parameter specified by the user.